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Ta BeO-OaTKIB 


The paper presents a model, which can be applied to Graphical User Interface testing. The model is dedicated 
especially to web pages and web applications created using Responsive web design. The procedure is based 
on expert method and cognitive walkthrough. The paper outlines contain the description of responsive web 
design, used methodology, GUI evaluation process and the application description. 

Key words: responsive web design, RWD, RWD, GUI usability, user experience. 


B ctatTbe mpescTaByieHa MOJesIb, KOTOpad MO2KeT ObITh MIpHMeHeHa K TecTHpOBaHUtO rpadwyeckoro uHTepdetica 
TIOJIb3OBaTeCJIA. Mogesb Tipe qHa3sHavena oco6eHHO ASIA BeO-cTpaHHIl Yu Be0-lmpuoxKeHuh, CO3HaHHBIX C HC- 
TIOJIb30BAHHeM TeEXHOJIOTMM afalTHBHOrO (OT3bIBUHBOrO) BeO-Av3aliHa. [Iporlenypa ocHOBaHa Ha MeTOJIe 9KCIepTHOTO 
WM KOrHHMTHBHOTO NOWAaroBoro o630pa. B ctTatbe v3102%KeHO OMCaHHe OT3bIBYMBOTO Be0-2M3aliHa, MCTIOsIb30BaHHOM 
MeTOAOJIO‘H, Mpolecca OLEHKM UHTepdelica Hu NpwIOKeHHA. 

KrovesBbie CJI0Ba: ajaliTHBHbIii BeO-zH3aiiH, RWD, RWD, yaoOctBo rpaduyeckoro uHTepdeiica, 
yoOcTBo padortsli. 


Y cTaTTi HaBeeHO MOJelIb, 1kKa MO*Ke OyTH 3aCTOCOBaHa JIA TecTyBaHHA rpadimHoro intepdelicy KopucTyBaya. 
OcoOmmBo MoOjeIb TIpu3Haiena Wid BeO-cTOpIHOK Ta BeO-10JaTKiB, CTBOPeHHX 3a JOMOMOTOIO BHKOpHCTaHHA 
TexHONOrilt ayanTHBHoro BeO-gH3aliny. [Tpouezypa 3acHOBaHa Ha MeTOJi CKCTepTHOrO i KOTHITHBHOTO MOKPOKOBOrO 
orsiagy. Y cTaTTi HaBeyeHo OMMc aflarTuBHoro BeO-7u3aliHy, MeTOJONOTIi, LO BUKOPHCTOBYETECA, TIPOLecy OMHKH 
TpadiqHoro intepdeticy kopuctyBaya iHTepdeticy Ta TonaTKa. 

K.no4osi c10Ba: aanTuBHHit BeO-qu3aiH, RWD, RWD, 3pyuHicte rpadisHoro inTepdeilicy, 
3py4HicTs poootu. 


Introduction 


Computer market gain certain stability. According research [1] most of people (in USA) 
own laptop or desktop computer. However, more and more people buy also smartphones and 
tablets. This trend of rapid adoption of smartphones and tablets will be continued in the near 
future. That is why it is very important to adapt web applications and web paged to require- 
ments of growing number of mobile users. Nowadays it is more important than never to design 
cross-channel websites [2], websites prepared to be accessed from a myriad of different devices. 


300 ISSN 1561-5359 «MckyccTBeHHbili HHTeWIeKT» 2013 Ne 3 


The Model of Quality Assessment of Automatically Adjusted Responsive Web Pages... | 4P 


Responsive web design (RWD) can be treated as a solution to the problem of necessity of 
automatically pages adjustment to all type of devices, like desktop computers, notebooks, tables 
or smartphones. The term “Responsive web design” was proposed by Ethan Marcotte in 2010 [3]. 
In 2012 “responsive design” was declared to be the second development trend of the year [4]. 

The goal of RWD is to deliver a quality experience to users no matter how large or small 
the display they use is. But RWD is not only about changing the way elements are displayed in 
a web page: RWD is also about how to provide easy reading and navigation with a minimum 
of resizing, panning, and scrolling across a wide range of different devices. Responsive web 
design mixes three different techniques, fluid grids, flexible images, and media queries, all of 
them based on the use of CSS (Cascading Stylesheets). 

GUI of the responsive design web page needs to be checked and tested by users working 
on different type of devices. The site designed in this way should be adjusted to different kind 
of screen resolutions (computer monitors, smart phones, tablets, etc). It should be readable 
with a minimum of resizing, panning, and scrolling. 

To check users’ viewing experience GUI quality evaluating methods might be used. 
During such case study users experience also needs to be regarded. 

Novice users interacting with a system for the first time prefer simple actions and 
ease of learning [5], but their behaviour and work quality changes as their experience with 
the system increases. This is an important point of view when evaluating GUI quality over 
time, but to setup such a study requires observation over time and lots of resources [6], 
which are often hard or even impossible to obtain. 

An expert analysis in combination with cognitive walkthrough seems to be the most 
reliable method in case of assessment of responsive design web page GUI quality. As it is 
shown in the studies [7] in most cases group of several experts using the adequate 
methodology is able to detect and correct over 85% of errors in software — this applies also 
to errors in GUI quality. It should be noted that testing can never completely identify all 
the defects within software [8]. 


1 Responsive application 


The example web page was created using Foundation framework dedicated to responsive 
web design. The web page should be readable with a minimum of resizing, panning, and scrol- 
ling. It was created to present user interface navigation mechanism and to get know users viewing 
experience. 

The web page is simple, it does not have extensive functionality. It contains several 
main sections: 

— Articles — extended thematic articles including photos. 

— Gallery — several galleries each containing a collection of photos 

— Form — typical web form composed of several questions. 

The home page contains news shortcuts and links to the remaining contents of the page. 

The menu section is placed on the left side of the page. Mobile and smartphone 
users, however, can see the menu at the top of the screen to make it more readable. 

Fig. 1 presents an example screen of the web page. The screen contains the section of 
articles. Fig. 1a) presents a screenshot taken on a computer and fig 1b) — on a mobile phone. 
The same page is displayed slightly different at those devices. The location and look of menu 
is different as well as the size of figures. 

Differences between web pages adjusted to computers and to mobile phones can be also 
seen in Fig. 2 and 3. Those figures present mockups of the two chosen web pages. Fig. 2a) and 
Fig. 3a) present mockup dedicated to computers and tablets with resolution of minimal width 
768 px, whereas Fig. 2b) and Fig. 3b) show mockups dedicated to smartfphones with small 
screens (maximal width: 767 px). 
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a) b) 
Figure 1 — Sample screenshot of the web page (source: own work) 


An Extended Big Title 


A Subtitle 


Lorem ipsum dolor sit amet, consectetur adipiscing elit. 
Vivamus vulputate, nunc vitae fringilla rhoncus, liguia quam 
scelerisque purus, nec volutpat sapien orci et leo. Aliquam 
erat volutpat. Quisque quis fringilla odio. Sed in est leo, id 
ornare leo. Sed ut est tellus. Aliquam ultrices massa ac sem 
molestie eu fringilla turpis laoreet. Integer ac convallis eros. In 
hac habitasse platea dictumst. Morbi eget augue a massa 
molestie viverra eu eget neque. Integer rutrum ultrices eros, 
non volutpat orci condimentum vel. Suspendisse sit amet 
commodo metus. Aliquam erat volutpat. 


Articles 


An Extended Big Title ag Tile 


BGGk to\home a 
asada Back to home 


VAVAVADN 
XX 


a) 


Figure 3 — Sample mockup of the gallery page (source: own work) 


302 «MckyCCTBeHHBIM HHTeJIeCKT» 2013 Ne3 


The Model of Quality Assessment of Automatically Adjusted Responsive Web Pages... | 4P 


It is important to design the application well. One need to consider different resolutions at 
this stage to be sure that the displayed page will be adjusted to desired resolutions and equip- 
ments. Properly done project can be easily transformed into an application using responsive web 
design. 

One should consider a place and layout of menu, construction of navigation and a way 
of pictures display. 

The page is always adjusted to the screen resolution, so horizontal scroll is never dis- 
played. The menu dedicated to mobile phones also differs — it has a form of the dropdown list 
and it is located at the top of the web page. Menu dedicated to higher screen resolution devices 
has a form of list and it is located at the left side of the web page. Also the size and location of 
figures is different. 


2 Applied methods 
2.1 Expert analysis 


Expert analysis is one of the most widely used method used in application testing. 
Experts, while using the application, check and mark the predefined areas. Those areas are 
defined to help to examine application compliance with interface design guidelines (such 
as Nielsen-Molich heuristics [9]) and to detect potential problems. 

Each of those predefined areas can be divided into several more detailed sub-areas. 
What is more, they usually have detailed questions assigned to help experts to cover more 
important aspects of GUI quality [10]. 

During the analysis the whole tested system is treated as a black box where only the inputs, 
outputs and specification are visible. Consequently, experts run only the functional testing. 
They focus on executing the system functions and accessing their input and output data [3]. 
Therefore system functionality is determined by observing the outputs to corresponding inputs 
and the implementation details are not considered [8]. 

Contrary to the widespread assumption, experts usually do not acquire better results 
in performing specific tasks in the tested system. They usually do not know the system 
before testing, they have no experience regarding this particular application. However, their 
expert status is based on extended experience with different kind of software. This fact, as 
proven by studies [11], [12], allows them to perform tasks faster than the novices and to 
spend less time handling the errors. Moreover, experts know user interface design fundamentals 
and principles. 


2.2 Cognitive walkthrough 


Simplified cognitive walkthrough is one of the expert methods of usability and GUI 
quality testing. It emphasizes the ease of interface learning as well as viewing experience 
during the initial contact with the system [10]. This methods might be successfully used in 
combination with the expert analysis method. 

This method is based on few tasks, which user will perform during his work with an 
application (e.g. making a purchase) [13]. Each task is divided into individual steps, which 
are analyzed according to the following questions: 

— Does the user know what to do during the analyzed step? 

— Ifthe action performed by user is correct, is he aware of it? 

— Ifthe action performed by user is correct, does he feel he is getting closer to reach 
the goal? 

The difficulty of each step is usually evaluated using a Likert scale of 1 to 5, with 1 
meaning ‘very easy’, 5 — ‘very difficult’. 
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In contrast to other GUI quality assessment methods, the main goal of the cognitive 
walkthrough is to study the flow of the processes undertaken by user. 

Cognitive walkthrough method has gained its popularity due to the ability of performing 
relatively quick analysis at low cost. Moreover, it may be used at different phases of the appli- 
cation design. However, cognitive walkthrough is sometimes regarded [5] as a method which de- 
tects potential problems with the interface instead of already existing problems. Therefore, it is 
also used in conjunction with other general tests as an addendum identifying potential problem 
areas. 


3. The testing method 


In order to obtain the results in a web page testing, the conglomerate of expert analysis 
and cognitive walkthrough was proposed. 

The expert analysis criteria are presented in Tablel and Table 2. Table 1 shows the 
detailed list of areas and subareas with questions assigned to each point. The group of main 
areas contains: Application interface, Navigation, Data structure and Data input. The list is 
a modified version of the list called “LUT list” which we proposed in [14]. Modification 
consists in adjusting sections concerning different devices and resolutions. Accordingly, 
Table 2 presents the grading scale used to assess each evaluated assessed area. 


Table 1 — The expert analysis list of predefined testing areas with questions assigned 
Application interface 
Layout Is the layout readable? 
Is it adjusted to different resolutions? 
Is it adjusted to mobile devices? 
Is it consistent? 
Does it support task implementation? 
Navigation 
Ease of use Is the access to all elements of menu easy and intuitive? 
Is the use of menu easy? 
Is the localization of menu intuitive? 


Menu hierarchy Isn’t the menu hierarchy too complicated? 
Navigation Is the navigation structure intuitive and understandable for users? 
structure Is the navigation well adjusted to the screen resolution? 

Is it well planned? 
Screen elements Do they support the navigation? 

Data structure 

Ease of use Is the access to all sections of an application easy and intuitive? 

Is the access to all functions of an application easy and intuitive? 
Information Isn’t the information hierarchy too complicated? 
hierarchy 
Information Is the information structure understandable? 
structure Is it consistent? 

Is it well planned? 

Data input 

Data Is the data validated by the form elements? 


Do the forms have elements acting as hints for the input data (e.g. on 
format or data range)? 

Can an average user fill in the form easily? 

Forms Are they designed in a readable way? 

Are they adjusted to the mobile devices? 

Do they allow user to input all of the necessary data? 
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Table 2 — The grading scale applied to the expert analysis list 


Grade Description 

1 Critical usability errors were observed, preventing normal usage or discouraging user from using 
the application. 

2 Serious usability issues were encountered, which may prevent most users from task realization. 

3 Minor usability issues were observed, which if accumulated may have negative impact on user performance. 

4 Single minor usability issues were observed, which may have negative impact on user work 
quality (e.g. poor readability). 

5) No usability issues influencing either user performance or work quality were identified. 


The cognitive walkthrough involves three scenarios containing tasks to perform for 
users. They are: 

— Run the web page, find the specified article 

- Find and display the specified photo in the gallery 

— Fill out the data form 

need to performed those task and answer several questions. They are: 

- The type of used device 

- The screen resolution 

- The type of web browser 

— Number of moves to accomplish the task 

— Number of mistakes 

= Additional remarks 

The results of analysis performed using those two methods will present the users 
experience and GUI quality of prepared responsive design web page. 


4 Proposed analysis process 


Responsive web page should be analyzed in a special way. The analysis process is 
recommended to be conducted by experts experienced with GUI quality analysis. 

The example of the analysis process could be described using the presented application. 
The analysis could be composed of two steps. The first one is an expert analysis, the second 
one — the cognitive walkthrough. Both analyses were performed using the survey. 

The first section enables to analyse the GUI quality, divided into four areas: Application 
interface, Navigation, Data structure and Data input. 

The second section regards the cognitive walkthrough composed of three tasks. They are 
presented in Fig.4. According the presented BPMN (Business Process Modelling Notation) 
process, users have to find the specified article, find and select the specified photo in one of 
galleries and also fill out and submit the data te 


Select section: Choose an 
articles article 
Select Choose the { Choose the 
section: specified specified 
galleries gallery photo 


On Select Fill the form |»! ( submit the 
section: form form 


Figure 4 — The testing process of the web page (source: own work) 

After fulfilling their tasks, users would be asked about their viewing experiences during 
accessing web page on different devices. What is more, they could give information about number 
of moves and mistakes they done. They should also share their opinion about responsive web 
design. 
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Summary 


Responsive web design enables to create web pages, which can be viewed on different 
devices in similar way. Of course, due to devices diversification, it is impossible to obtain exactly 
the same web page look and user experience on all devices. However, the exact page look is no 
more desired result. The aim is to adapt the page to different users’ scenarios, not for different 
screens. Website needs to adapt to users’ interaction patterns changes. 

Responsive web design helps to obtain this goal. Users do not need to install anything 
extra or to set any properties. They can use one common interface. However, one should be 
aware that the only thing certain about the future is that web design is not predictable. Its 
development will alway be adjusted to the technology and device trends. 

Future works will include a case study on presented responsive web design project. The 
analysis process will regard both methods: Expert analysis and Cognitive walkthrough. 
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eK OMDANDNN 


RESUME 
M. Plechawska-Wojcik 


The Model of Quality Assessment of Automatically Adjusted 
Responsive Web Pages and Applications 


The main goal of the presented case study was to assess capabilities of Responsive 
Web Design (RWD) and to determine the user experience of RWD web pages. The case 
study was based on two methods (expert analysis and cognitive walkthrough) applied on 
web experienced users using different devices like computers, smart phones, tablets. 

Obtained results confirm the effectiveness of RWD. Low number of errors and 
quickly completed tasks prove that RWD can be successfully applied in practice. 


The paper is received by the edition 18.04.2013. 
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